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Assessment tool 
NTM-A Pilot 


Introduction and Prior Research 


I devoted years of my life to t his work and want to stay engaged with it; my passion is a strange 
bedfellow with collaboration.! 


(b)(3). (b)(6), (b)(7)(C) 


is on complex tasks or programs 

til individual missions, that are working with the inter/intra-agency 
and how each are impacted by the others. This led me to discover a profound difference at the 
institutional level. I then started to work on how to align efforts at the strategic levels for increased 
synergy. There was also a profound difference on how to get past individual factors like leadership 
skills or if personnel knew another actor personally say from school. 



that have multiple pl:iyers. \vi 


"You don't want to compromise the integrity of individual actors. Civ-mil relations are already 
so fraught and so important to both of their individual agendas, especially in a place like 
Afghanistan." 
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The chair of my dissertation committee actually rejected my initial, broader research and so 1 
distilled it down to the fine print My definition of coordination is broader, not just active 
coordination. There is self-organized coordination that can emerge and can be effective for strategic 
alignment and improves institutional synergy. 

"Vfe can set conditions to help coordination evolve organically." 


Current Research 


The past eight years I have been the lead researcher for several OSD research grants. The work 
looks at methodologies, tools, and meta-data to support theories of change or monitoring and 
evaluation efforts. We piloted it with State/NTM-A and it would be interesting so to you [SIGAR 
staff] the tools and applicability. This tool helps us look at these wicked problems and their iterative 
nature by continually updating goals and strategies. 


The hyper-local focus is nested within the complex and built at a national-level scale. The tool 
would be built in with the headquarters (NSC) levels and with grand outcomes. Of most interest to 
me is the research and mapping out of how coordination played out at higher levels and how it 
played out horizontally and we where we can best leverage or intervene. To look at this I did 
several high level interviews including with Lute at the NSC and can possibly distill interview notes 
and share them. I had voluminous notes and basically have verbatim notes, but will have to ask the 
interviewees if it would be alright to share. 
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I left government andC 


_called and asked to use me for a proposal on an OSD 

innovation grant and they really convinced be because of the topic matter. 1 could earn money and 
also consult on research directly related to my Ph.D. The line of funding recently whittled down and 
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the next faucet is unknown because the lines of funding are shifting._ 

roughly eight grants over eight years of varying sizes. Some small, like Phase I, or Phase II (~two 
years, $1.5 million) and some larger, like Phase III (IDIQ, pre-competed). I worked with the Army 


won 
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I consult and 


Corps and O.N.R.._ 

lead the effort alongside the CEO. We also have some IT staff for support What we developed had a 
light methodology, but heavy process and had a joint M&E process. It was a light methodology 
because tools are to be used to support and enhance, not impose. This was cool because it can be 
used between different planning systems or cultures - joint assessment planning is possible in this 
environment It is a light guide to action, not meant to impose. 


Assessment Tool IRTAP?! 

This tool suite links outputs to outcomes and helps to visualize the theory of change - it was 
as 3 mchronous. We map it all out For example, if we build a school in Afghanistan, will that increase 
female political participation? Our environment allows to show these steps explicitly. We use logic 
boxes which are tagged by agency which oversees it all and organizes metrics. It is also linked to 
timetables and allows one to tailor a metric library. It show trend lines starting from the baseline (it 
does not look at causality), and does some h 3 q)othesis testing. It is organized into different levels for 
different actors, such as a Comm ander. Things are color coded and soon we will incorporate 
causality.is incentivized to do pilot projects by OSD because the money for 
pilots is usually matched by OSD. The tool provides immediate results and can have tailored 
visualizations and tailored guidance. 


NTM-A Pilot 

With the NTM-A pilot there were a bunch of different plans that were inter-related but not 
connected and many of the plans were simply saved in Microsoft Word formats. Within hours of 
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being in Kabul, people were leaning in and started to see how this tool can provide a better 
planning process. The tool is meant to capture necessary/sufficient conditions, how to redefine/re- 
test, and organize mental models. We want to capture theories of changes by agency and their 
necessary or sufficient conditions to achieve goals and how those things are measured. 

“For the NTM~A pilot, we got traction, but people were unwilling to put down anything in the 
theory of change that was outside of their purview because they might be held accountable for 
it" 

One example would be improving security in Afghanistan. There were many different lines of effort 
for DDR but they would only incorporate lines of effort for DDR that they owned the funding for, 
but not other efforts like from the U.N., NGOs etc. If we could include all the outcomes it might show 
that ±e theory of change was wrong or not enough. 

We use meta-data behind our causal arrows (historical or anecdotal data) to show why you think 
this [theory of change] works. 

Next Steps; 

• Setup a web call or possible in-person meeting to show some slides or the tool itself. 





